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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
BEFORE THE BOARD OF PATENT APPEALS AND INTERFERENCES 



Appellant: Kasutani et al. 

Serial No.: 10/516,086 

Filed: November 29, 2004 

For: IMAGE SYSTEM 

Conf.No.: 9654 

Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA. 22313-1450 

APPEAL BRIEF 

I. Statement of Real Party in Interest 

The real party in interest of the present application is NEC Corporation, the assignee of 
the entire right, title and interest in the above-identified patent application. 



Examiner: Jayesh A. Patel 
Art Unit: 2624 
Docket: 18463 
Dated: June 23, 2008 



CERTIFICATE OF ELECTRONIC FILING 

I hereby certify that this correspondence is being deposited with the United States 
Patent & Trademark Office via Electronic Filing through the United States Patent and Trademark 
Office e-business website, on June 23, 2008. 

Dated: June 23, 2008 
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II. Statement of Related Appeals and Interferences 

No other appeals and interferences are known which directly affect, or will be directly 
affected by, or have a bearing on, the disposition of the pending appeal. 

III. Statement of Status of Claims 

Claim 1 stands rejected based on 35 U.S.C. § 102(b) as allegedly anticipated by U.S. 
Patent No. 6,052,492 issued to Bruckhaus. 

Claim 2 stands rejected based on 35 U.S.C. § 102(b) as allegedly anticipated by U.S. 
Patent No. 6,052,492 issued to Bruckhaus. 

Claim 3 stands rejected based on 35 U.S.C. § 102(b) as allegedly anticipated by U.S. . 
Patent No. 6,052,492 issued to Bruckhaus. 

Claim 4 stands rejected based on 35 U.S.C. § 103(a) as allegedly obvious over Bruckhaus 
in view of Nagasaka et ah 

Claim 5 stands rejected based on 35 U.S.C. § 103(a) as allegedly obvious over Bruckhaus 
in view of Nagasaka et al. 

Claim 6 Canceled 

Claim 7 Canceled 

Claim 8 Canceled 

Claim 9 stands rejected based on 35 U.S.C. § 102(b) as allegedly anticipated by U.S. 
Patent No. 6,052,492 issued to Bruckhaus. 

Claim 10 stands rejected based on 35 U.S.C. § 102(b) as allegedly anticipated by U.S. 
Patent No. 6,052,492 issued to Bruckhaus. 

Claim 1 1 Withdrawn 
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Claim 12 stands rejected based on 35 U.S.C. § 102(e) as allegedly anticipated by U.S. 
Patent No. 6,400,890 issued to Nagasaka et al. 

Claim 13 stands rejected based on 35 U.S.C. § 102(e) as allegedly anticipated by U.S. 
Patent No. 6,400,890 issued to Nagasaka et al. 

Claim 14 stands rejected based on 35 U.S.C. § 102(e) as allegedly anticipated by U.S. 
Patent No. 6,400,890 issued to Nagasaka et al. 

Claim 15 stands rejected based on 35 U.S.C. § 102(e) as allegedly anticipated by U.S. 
Patent No. 6,400,890 issued to Nagasaka et al. 

Claim 16 Canceled 

Claim 17 Canceled 

Claim 18 Canceled 

Claim 19 Withdrawn 

Claim 20 Withdrawn 

Claim 21 Canceled 

Claim 22 stands rejected based on 35 U.S.C. § 102(e) as allegedly anticipated by U.S. 
Patent No. 6,400,890 issued to Nagasaka et al. 

Claim 23 stands rejected based on 35 U.S.C. § 102(e) as allegedly anticipated by U.S. 
Patent No. 6,400,890 issued to Nagasaka et al. 

Claims 1 - 5, 9, 10, 12 - 15 and 22 - 23 are under appeal. 

IV. Statement of Status of Amendments 

No amendments were made subsequent to the final rejection. 
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V. Summary of Claimed Subject Matter 

The invention with respect to claim 1 comprises an image description system comprising: 
a feature extracting unit [101] extracting color layout features from respective of frames of an 
image sequence (e.g., See: FIG. 3 and 1 0; and page 23, lines 4 - 6); and a representative feature 
calculating unit [102] calculating a representative color layout feature representative of said 
image sequence from a group of said color layout features of all frames extracted by said feature 
extracting unit [101] (e.g., See: FIG. 3 and 10; and page 23, line 25 - page 24, line 2). 

The invention with respect to claim 2 comprises an image description system comprising: 
a feature extracting unit [101] extracting color layout features from respective of frames of an 
image sequence (e.g., See: FIG. 3 and 10; and page 23, lines 4 - 6); a representative feature 
calculating unit [102] calculating a representative color layout feature representative of said 
image sequence from a group of said color layout features extracted by said feature extracting 
unit [101] (e.g., See: FIG. 3 and 10; and page 23, line 25 - page 24, line 2); and a representative 
layout feature storage unit [103] storing said representative color layout feature calculated by said 
representative feature calculating unit [102] (e.g., See: FIG. 3 and 10 - 12; and page 31, lines 14 
-17). 

The invention with respect to claim 3 comprises an image description system comprising: 
a feature extracting unit [101] extracting color layout features from respective of frames of an 
image sequence (e.g., See: FIG. 3 and 10; and page 23, lines 4 - 6); a representative feature 
calculating unit [102] calculating a representative color layout feature representative of said 
image sequence from a group of said color layout features extracted by said feature extracting 
unit [101] (e.g., See: FIG. 3 and 10; and page 23, line 25 - page 24, line 2); a representative 
layout feature storage unit [103] storing said representative color layout feature calculated by said 
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representative feature calculating unit [102] (e.g., See: FIG. 3 and 10 - 12; and page 31, lines 14 
- 17); and a layout feature group storage unit [104] storing said group of color layout features 
calculated by said feature extracting unit [101] (e.g., See: FIG. 3 and 10; and page 31, lines 17 - 
20). 

The invention with respect to claim 4 comprises the elements identified with respect to 
claim 1 above, and further wherein said representative feature calculating unit [102] calculates 
average values of respective elements of a color layout feature extracted by said feature 
extracting unit [101] as a said representative color layout feature (e.g., See: FIG. 4 and 6; page 
14, lines 8 - 13; and page 24, line 8 - page 25, line 13). 

The invention with respect to claim 5 comprises the elements identified with respect to 
claim 1 above, and further wherein said representative feature calculating unit [102] rearranges 
values of respective elements of a color layout feature extracted by said feature extracting unit 
[101] in any of ascending order and descending order, and calculates a median as said 
representative color layout feature (e.g., See: FIG. 5 and 7; page 14, lines 14 - 20; and page 25, 
line 14 -page 26, line 19). 

The invention with respect to claim 9 comprises an image description software product 
embodied on a computer-readable medium, said software product having instructions executable 
on a computer for performing the steps comprising: extracting color layout features from 
respective of frames of an image sequence; and calculating a color layout feature representative 
of said image sequence from a group of said color layout features extracted by said feature 
extracting function (e.g., See: page 15, lines 4 - 12). 
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The invention with respect to claim 10 comprises an image description method 
comprising: extracting color layout features from respective of frames of an image sequence; and 
calculating a color layout feature representative of said image sequence from a group of said 
color layout features extracted in said extracting (e.g., See: page 15, lines 13 - 21). 

The invention with respect to claim 12 comprises an image identification system 
comprising: a representative layout feature storage unit [103] storing a color layout feature 
representative of an image sequence as a representative color layout feature; a feature extracting 
unit [105] extracting color layout features from respective of frames of a query image sequence; a 
representative feature calculating unit [106] calculating a representative color layout feature 
representative of said query image sequence from a group of said color layout features extracted 
by said feature extracting unit [105]; and an image sequence selecting unit [107] selecting a 
sequence which resembles said query image sequence by comparing said representative color 
layout feature calculated by said representative feature calculating unit [106] with said 
representative color layout feature stored in said representative layout feature storage unit [103] 
(e.g., See: FIG. 3 and 11; page 15, line 22 - page 16, line 5; and page 31, line 26 - page 32, line 
20). 

The invention with respect to claim 13 comprises an image identification system 
comprising: a representative layout feature storage unit [1 03] storing a color layout feature 
representative of an image sequence as a representative color layout feature; a layout feature 
group storage unit [104] storing color layout features associated with respective of frames of said 
image sequence; a feature extracting unit [105] extracting color layout features from respective 
of frames of a query image sequence; a representative feature calculating unit [106] calculating a 
representative color layout feature representative of said query image sequence from a group of 
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said color layout features extracted by said feature extracting unit [105]; an image sequence 
selecting unit [107] selecting an image sequence which resembles said query image sequence by 
comparing said representative color layout feature calculated by said representative feature 
calculating unit [106] with said representative color layout feature stored in said representative 
layout feature storage unit [103]; and an identification unit [108] matching said group of color 
layout features extracted by said feature extracting unit [105] against said color layout features 
stored in said layout feature group storage unit [104] as to said image sequence selected by said 
image sequence selecting unit [107] (e.g., See : FIG. 3; page 16, lines 6 - 23; and page 31, line 26 
- page 33, line 8). 

The invention with respect to claim 14 comprises the elements identified with respect to 
claim 12 above, and further wherein said representative feature calculating unit [106] calculates 
average values of respective elements of a color layout feature extracted by said feature 
extracting unit [105] as said representative color layout feature (e.g., See: page 17, lines 21 - 26). 

The invention with respect to claim 1 5 comprises the elements identified with respect to 
claim 12 above, and further wherein said representative feature calculating unit [106] rearranges 
values of respective elements of a color layout feature extracted by said feature extracting unit 
[105] in any of ascending order or descending order, and calculates a median as said 
representative color layout feature (e.g., See: page 17, line 27 - page 18, line 5). 

The invention with respect to claim 22 comprises the elements identified with respect to 
claim 13 above, and further wherein said representative feature calculating unit [106] calculates 
average values of respective elements of a color layout feature extracted by said feature 
extracting unit [105] as said representative color layout feature (e.g., See: page 17, lines 21 - 26). 
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The invention with respect to claim 23 comprises the elements identified with respect to 
claim 13 above, and further wherein said representative feature calculating unit [106] rearranges 
values of respective elements of a color layout feature extracting unit [105] in any of ascending 
order or descending order, and calculates a median as said representative color layout feature 
(e.g., See: page 17, line 27 - page 18, line 5). 

VI. Statement of Grounds of Rejection to be Reviewed on Appeal 

1. Rejection under 35 U.S.C. § 102(b) as allegedly anticipated by U.S. Patent No. 
6,052,492 issued to Bruckhaus. 

2. Rejection under 35 U.S.C. § 102(e) as allegedly anticipated by U.S. Patent No. 
6,400,890 issued to Nagasaka et al. 

3. Rejection under 35 U.S.C. § 103(a) as allegedly obvious over Bruckhaus in view of 
Nagasaka et al. 

VII. Statement of Argument 

A. Rejection of Claims 1 - 3 and 9 - 10 Under 35 U.S.C. $ 102(b) 

The rejection of Claims 1 - 3 and 9 - 10 is based on the allegedly anticipatory teachings 
of Bruckhaus. The Examiner contends that Bruckhaus discloses an image description system 
(FIG.l - 3) comprising a feature extracting unit (FIG. 2, 215) extracting color layout features 
from respective frames of an image sequence; a representative feature calculating unit (FIG. 2, 
230) calculating a representative color layout feature representative of the image sequence from a 
group of color layout features extracted by the feature extracting unit. (See: col. 4, lines 35-56, 
col. 8, lines 24-41 and col. 10, lines 25-46). 

Bruckhaus discloses identifying various features in an image, such as a basketball, shirts, 
etc. Even though color information may play a part in the identification process, Bruckhaus does 
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not, however use this color information in calculating a representative color layout feature 
representative of the image sequence from a group of layout features of all frames extracted. 
Rather, Brucfchaus discloses that the unit extractor extracts all the units, i.e., an integral set of 
pixels representing a single physical object in a frame. Of these extracted units a plurality of the 
most predominant units are composited into a representative image. (See : FIG. 4 - 6b). 

Regardless of whether Bruckhaus discloses attributes such as color brightness and 
motion, no where in Bruckhaus is a representative feature calculating unit disclosed for 
calculating a representative color layout feature representative of said image sequence from a 
group of said layout features of all frames extracted by said feature extracting unit. 

Color layout feature within the context of the present invention has a specific meaning, 
which must be considered when asserting that a prior art reference discloses such a feature. 
Regarding the meaning of color layout feature within this context, Appellants point to FIG. 6 and 
FIG. 7 for clear examples of color layout values. These color layout values are examples of color 
layout features in the present invention, and thus provide a definition of the term "color layout 
features" as used throughout the disclosure and claims. Also, as shown in FIG. 6 and 7, a 
representative color layout feature representative of an image sequence from a group of layout 
features of all frames in the image sequence are shown. (See : Total and Average rows in FIG. 6; 
and Median row in FIG. 7). It is evident from Appellants' disclosure that the color layout features 
are color component values, such as luminance and color difference. 

Moreover, Appellants' disclosure provides clear definition of the term "color layout 
feature" as applied throughout the specification and claims. Specifically, the specification recites: 
"[TJhese layout features are defined in the International Standard ISO/IEC 15838-3 MPEG-7 
VISUAL. . ." (See: page 23, lines 4 - 24). 
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In contrast, Bruckhaus explicitly defines a unit as ". . .an integral set of pixels representing 
a single physical object in the frame." (See : col. 4, lines 35 - 50). 

Unlike Bruckhaus, which generates a representative image from representative shapes 
(i.e., physical objects) in the frames, the present invention generates a representative color layout 
feature, such as a total color feature value, average color feature value or a median color feature 
value, where these color feature values are essentially color component values as defined in the 
above-mentioned ISO/IEC standard. Rather than being a combination of shapes representing the 
image sequence, the representative color layout feature of the present invention is a single color 
layout feature representative of the color layout features in the image sequence. 

Given the fundamental difference in the meaning of "unit" as defined in Bruckhaus and 
"layout feature" as defined in Appellants' disclosure, one of ordinary skill in the art would not 
equate one with the other. Therefore, Bruckhaus fails to properly anticipate ". . .extracting color 
layout features from respective of frames of an image sequence; and. . . calculating a 
representative color layout feature representative of said image sequence from a group of said 
color layout features of all frames extracted...", recited in Claims 1 - 3 and 9-10. 
B. Rejection of Claims 12 - 15. 22 and 23 Under 35 U.S.C. § 102fe^> 

The rejection of Claims 12 - 15, 22 and 23 is based on the allegedly anticipatory 
teachings of Nagasaka et al. As with Bruckhaus, Nagasaka et al. fails to disclose calculating a 
representative color layout feature representative of the image sequence from a group of color 
layout features of all frames extracted. Rather, Nagasaka discloses a frame feature extractor that 
extracts a feature from each frame and assigns the extracted feature to represent the individual 
frame from which it was extracted. Thus, the extracted feature is not calculated to represent an 
image sequence. 
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In addition, the features that are extracted are not color layout features, as described 
above and defined within Applicants' disclosure. Rather as described in col. 14, line 37 - col. 15, 
line 30, these features are physical objects, existence of a camera work, existence of a special 
effect, existence of a title, etc. No mention is made, however, of color layout features in 
Nagasaka et al. 

Therefore, Nagasaka et al. does not disclose each and every element recited in the present 
claims. Consequently, Appellants submit that Nagasaka et al. fails to properly anticipate all the 
features recited in Claims 12 - 15, 22 and 23. 
C. Rejection of Claims 4 and 5 Under 35 U.S.C. S 103(a) 

The rejection of Claims 4 and 5 is based on the combined teachings of Bruckhaus and 
Nagasaka et al. However, as discussed above, neither reference discloses calculating a 
representative color layout feature representative of the image sequence from a group of color 
layout features of all frames extracted. Therefore, the references, taken alone or in any proper 
combination fail to disclose the features recited in Claim 1 from which Claims 4 and 5 depend. 

VIII. CONCLUSION 

It is clear that all of the limitations of Claims 1 - 3, 9, 10, 12 - 15, 22 and 23 are not 
taught or suggested by the references of Bruckhaus and Nagasaka et al. Accordingly, Appellants 
submit that the Examiner has failed to establish anticipation of Claims 1 - 3, 9, 10, 12-15, 22 
and 23 based on the prior art references individually. 

Moreover, Appellants respectfully submit, with respect to Claims 4 and 5, that the 
Examiner has not met the burden of establishing a prima facie case of obviousness based on the 
prior art, as required by 35 U.S.C. § 103(a); no objective teaching in Bruckhaus and Nagasaka et 
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al., individually or in combination, would lead an individual of ordinary skill in the art to produce 
the present invention. 

The above arguments establish that Claims 1 - 5, 9, 10, 12 - 15, 22 and 23 on appeal are 
patentable over Bruckhaus and Nagasaka et al., individually or in any proper combination 
thereof. In view of the remarks set forth in this Appeal Brief, Appellants respectfully request that 
the rejections under 35 U.S.C. § 102(b), 35 U.S.C. § 102(e) and 35 U.S.C. § 103(a) citing the 
aforementioned references made in the Final Rejection dated October 19, 2007, and in the 
Advisory Action of March 20, 2008, be reversed by the Board of Patent Appeals and 
Interferences. 



SCULLY, SCOTT, MURPHY & PRESSER 
400 Garden City Plaza - Ste. 300 
Garden City, New York 11530 
(516) 742-4343 

PJE/DAT 



Rest 




illy submitted, 



Paul J. Lsatto, Jr. 
Registration No. 30.749 
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CLAIMS APPENDIX 

1 . (Previously Presented) An image description system comprising: 

a feature extracting unit extracting color layout features from respective of frames of an 
image sequence; and 

a representative feature calculating unit calculating a representative color layout feature 
representative of said image sequence from a group of said color layout features of all frames 
extracted by said feature extracting unit. 

2. (Original) An image description system comprising: 

a feature extracting unit extracting color layout features from respective of frames of an 
image sequence; 

a representative feature calculating unit calculating a representative color layout feature 
representative of said image sequence from a group of said color layout features extracted by said 
feature extracting unit; and 

a representative layout feature storage unit storing said representative color layout feature 
calculated by said representative feature calculating unit. 

3 . (Original) An image description system comprising: 

a feature extracting unit extracting color layout features from respective of frames of an 
image sequence; 

a representative feature calculating unit calculating a representative color layout feature 
representative of said image sequence from a group of said color layout features extracted by said 
feature extracting unit; 
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a representative layout feature storage unit storing said representative color layout feature 
calculated by said representative feature calculating unit; and 

a layout feature group storage unit storing said group of color layout features calculated 
by said feature extracting unit. 

4. (Original) The image description system according to claim 1 , wherein said representative 
feature calculating unit calculates average values of respective elements of a color layout feature 
extracted by said feature extracting unit as a said representative color layout feature. 

5. (Original) The image description system according to claim 1 , wherein said representative 
feature calculating unit rearranges values of respective elements of a color layout feature 
extracted by said feature extracting unit in any of ascending order and descending order, and 
calculates a median as said representative color layout feature. 

6. (canceled) 

7. (canceled) 

8. (canceled) 

9. (Previously Presented) An image description software product embodied on a computer- 
readable medium, said software product having instructions executable on a computer for 
performing the steps comprising: 

extracting color layout features from respective of frames of an image sequence; and 
calculating a color layout feature representative of said image sequence from a group of 
said color layout features extracted by said feature extracting function. 
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10. (Original) An image description method comprising: 

extracting color layout features from respective of frames of an image sequence; and 
calculating a color layout feature representative of said image sequence from a group of 
said color layout features extracted in said extracting. 

11. (Withdrawn) An image identification system comprising: 

a representative layout feature storage unit storing a color layout feature representative of 
an image sequence as a representative color layout feature; and 

an image sequence selecting unit selecting an image sequence which resembles a query 
image sequence by comparing said representative color layout feature of said query image 
sequence stored in said representative layout feature storage unit with said representative color 
layout feature of a registered image sequence stored in said representative layout feature storage 
unit, 

12. (Original) An image identification system comprising: 

a representative layout feature storage unit storing a color layout feature representative of 
an image sequence as a representative color layout feature; a feature extracting unit extracting 
color layout features from respective of frames of a query image sequence; 

a representative feature calculating unit calculating a representative color layout feature 
representative of said query image sequence from a group of said color layout features extracted 
by said feature extracting unit; and 

an image sequence selecting unit selecting a sequence which resembles said query image 
sequence by comparing said representative color layout feature calculated by said representative 
feature calculating unit with said representative color layout feature stored in said representative 
layout feature storage unit. 
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13. (Original) An image identification system comprising: 

a representative layout feature storage unit storing a color layout feature representative of 
an image sequence as a representative color layout feature; a layout feature group storage unit 
storing color layout features associated with respective of frames of said image sequence; 

a feature extracting unit extracting color layout features from respective of frames of a 
query image sequence; a representative feature calculating unit calculating a representative color 
layout feature representative of said query image sequence from a group of said color layout 
features extracted by said feature extracting unit; 

an image sequence selecting unit selecting an image sequence which resembles said query 
image sequence by comparing said representative color layout feature calculated by said 
representative feature calculating unit with said representative color layout feature stored in said 
representative layout feature storage unit; and 

an identification unit matching said group of color layout features extracted by said 
feature extracting unit against said color layout features stored in said layout feature group 
storage unit as to said image sequence selected by said image sequence selecting unit. 

14. (Original) The image identification system according to claim 12, wherein said representative 
feature calculating unit calculates average values of respective elements of a color layout feature 
extracted by said feature extracting unit as said representative color layout feature. 

15. (Original) The image identification system according to claim 12, wherein said representative 
feature calculating unit rearranges values of respective elements of a color layout feature 
extracted by said feature extracting unit in any of ascending order or descending order, and 
calculates a median as said representative color layout feature. 
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16. (canceled) 

17. (canceled) 

18. (canceled) 

19. (Withdrawn) An image identification software product executable on a computer comprising: 

computer readable code configured to cause said computer to store a color layout feature 
representative of an image sequence as a representative color layout feature in a storage unit of 
said computer; and 

computer readable code configured to cause said computer to select an image sequence 
which resembles a query image sequence by comparing said representative color layout feature of 
said query image sequence stored in said storage unit with said representative color layout feature 
of a registered image sequence stored by in said storage unit. 

20. (Withdrawn) An image identification method comprising: 

storing a color layout feature representative of an image sequence as a representative 
color layout feature; and 

selecting an image sequence which resembles a query image sequence by comparing said 
representative color layout feature of said query image sequence stored in said storing with said 
representative color layout feature of a registered image sequence stored in said storing. 

21. (canceled) 

22. (Original) The image identification system according to claim 13, wherein said representative 
feature calculating unit calculates average values of respective elements of a color layout feature 
extracted by said feature extracting unit as said representative color layout feature. 
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23. (Original) The image identification system according to claim 13, wherein said representative 
feature calculating unit rearranges values of respective elements of a color layout feature 
extracting unit in any of ascending order or descending order, and calculates a median as said 
representative color layout feature. 
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EVIDENCE APPENDIX 
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RELATED PROCEEDINGS APPENDIX 

None 
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